Search for: All records

Abstract BackgroundSystematic literature reviews (SLRs) are foundational for synthesizing evidence across diverse fields and are especially important in guiding research and practice in health and biomedical sciences. However, they are labor intensive due to manual data extraction from multiple studies. As large language models (LLMs) gain attention for their potential to automate research tasks and extract basic information, understanding their ability to accurately extract explicit data from academic papers is critical for advancing SLRs. ObjectiveOur study aimed to explore the capability of LLMs to extract both explicitly outlined study characteristics and deeper, more contextual information requiring nuanced evaluations, using ChatGPT (GPT-4). MethodsWe screened the full text of a sample of COVID-19 modeling studies and analyzed three basic measures of study settings (ie, analysis location, modeling approach, and analyzed interventions) and three complex measures of behavioral components in models (ie, mobility, risk perception, and compliance). To extract data on these measures, two researchers independently extracted 60 data elements using manual coding and compared them with the responses from ChatGPT to 420 queries spanning 7 iterations. ResultsChatGPT’s accuracy improved as prompts were refined, showing improvements of 33% and 23% between the initial and final iterations for extracting study settings and behavioral components, respectively. In the initial prompts, 26 (43.3%) of 60 ChatGPT responses were correct. However, in the final iteration, ChatGPT extracted 43 (71.7%) of the 60 data elements, showing better performance in extracting explicitly stated study settings (28/30, 93.3%) than in extracting subjective behavioral components (15/30, 50%). Nonetheless, the varying accuracy across measures highlighted its limitations. ConclusionsOur findings underscore LLMs’ utility in extracting basic as well as explicit data in SLRs by using effective prompts. However, the results reveal significant limitations in handling nuanced, subjective criteria, emphasizing the necessity for human oversight. 

Human behavior shapes epidemic trajectories, evolving as individuals reassess risks over time. Our study closes the loop between epidemic status, individual risk assessments, and interactions. We developed an agent-based model where the individuals can alter their decisions based on perceived risks. In our model, agents’ perceived risk is proxied by their full awareness of actual risks, such as the probability of infection or death. We conducted several simulations of COVID-19 spread for a large metropolitan city akin to New York City, covering the period from December 2020 to May 2021. Our model allows residents to decide daily on traveling to crowded city areas or stay in neighborhoods with relatively lower population density. Our base run simulations indicate that when individuals assess their own risk and understand how diseases spread, they adopt behaviors that slow the spread of virus, leading to fewer cumulative cases and deaths but extending the duration of the outbreak. This model was then simulated with various vaccination strategies such as random distribution, prioritizing older individuals, high-contact-rate individuals, or crowded area residents, all within a risk-response behavioral framework. Results show that, in the presence of agents’ behavioral response, there is only a marginal difference across different vaccination strategies. Specifically, vaccination in crowded areas slightly outperformed other vaccination strategies in reducing infections and prioritizing the elderly was slightly more effective in decreasing deaths. The lack of a universally superior vaccination strategy comes from the fact that lowering a risk leads to more risky behavior which partly compensates for vaccination effects. The comparable outcomes of random versus targeted vaccinations highlight the importance of equitable distribution as another key focus in pandemic responses. 

In the first two years of the COVID-19 pandemic, per capita mortality varied by more than a hundredfold across countries, despite most implementing similar nonpharmaceutical interventions. Factors such as policy stringency, gross domestic product, and age distribution explain only a small fraction of mortality variation. To address this puzzle, we built on a previously validated pandemic model in which perceived risk altered societal responses affecting SARS-CoV-2 transmission. Using data from more than 100 countries, we found that a key factor explaining heterogeneous death rates was not the policy responses themselves but rather variation in responsiveness. Responsiveness measures how sensitive communities are to evolving mortality risks and how readily they adopt nonpharmaceutical interventions in response, to curb transmission. We further found that responsiveness correlated with two cultural constructs across countries: uncertainty avoidance and power distance. Our findings show that more responsive adoption of similar policies saves many lives, with important implications for the design and implementation of responses to future outbreaks. 

The drug shortage crisis in the last decade not only increased health care costs but also jeopardized patients’ health across the United States. Ensuring that any drug is available to patients at health care centers is a problem that official health care administrators and other stakeholders of supply chains continue to face. Furthermore, managing pharmaceutical supply chains is very complex, as inevitable disruptions occur in these supply chains (exogenous factors), which are then followed by decisions members make after such disruptions (internal factors). Disruptions may occur due to increased demand, a product recall, or a manufacturer disruption, among which product recalls—which happens frequently in pharmaceutical supply chains—are least studied. We employ a mathematical simulation model to examine the effects of product recalls considering different disruption profiles, e.g., the propagation in time and space, and the interactions of decision makers on drug shortages to ascertain how these shortages can be mitigated by changing inventory policy decisions. We also measure the effects of different policy approaches on supply chain disruptions, using two performance measures: inventory levels and shortages of products at health care centers. We then analyze the results using an approach similar to data envelopment analysis to characterize the efficient frontier (best inventory policies) for varying cost ratios of the two performance measures as they correspond to the different disruption patterns. This analysis provides insights into the consequences of choosing an inappropriate inventory policy when disruptions take place.